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This paper suggests rethinking the doctoral examination process in Australia to enable a closer alignment with the aims of the learning 
programme. The doctoral examination processes generally aim to assess the candidate’s research capability, the quality and originality of 
the candidate’s contribution to knowledge, and to authenticate that the work undertaken is in fact that of the candidate. However, based 
on a review of the Australian system for examining it is clear that the process does not fully align with the aims of the current doctorate 
and that modest changes could remedy this situation. The paper concludes with some suggestions for change which might enhance the 
assessment process of Australia’s highest academic award and issues which need to be considered. 


Introduction 

The doctoral examination process is critical in that it 
is the culmination of at least three to four years of full- 
time, or six to eight years part-time, work of a candidate, 
supervisor(s) and colleagues. Furthermore, of all the 
higher education awards, it could be argued that the 
doctorate is the most portable and widely recognised. 
As Lovitts (2007, p. 23) suggests, ‘Unlike other levels of 
education that have been struggling to specify what 
students should know and be able to do as a result of 
their educational experience and devise authentic or 
true tests of that knowledge and those skills, doctoral 
education has a true test — the dissertation’. 

Yet the examination systems for the doctorate vary 
widely between countries (ffartley, 2000). Perhaps the 
two extremes are the USA where examination is gener- 
ally a fully ‘internal’ matter, that is, with the candidate’s 
dissertation committee and host department making 
the decision regarding the quality and acceptability of 
the dissertation, and Australia, where the examination 


of the doctorate is very explicitly an ‘external’ matter 
with recommendations regarding quality made out- 
side the institution. Regardless of the system, ensuring 
that the process addresses all the main issues involved 
in the doctoral experience is an important educational, 
as well as organisational, responsibility. 

For the purposes of this paper a curriculum, rather 
than policy perspective is adopted in the discussion of 
a number of key issues related to the examination of 
the doctorate. Using curriculum’ in its broadest sense 
as a way of examining the issues provides a way of 
viewing the candidate, the supervisor, the learning, and 
the assessment process within the context of the disci- 
pline, the institutions and global structures. The paper 
begins with a necessarily brief description of the Aus- 
tralian examination system which produces a Dr’ com- 
parable with any other ‘Dr’ worldwide and then builds 
on the growing research in the field of research educa- 
tion to discuss the following: 

• What is the purpose of the doctorate? 

• What is the object of assessment? 
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• Who should assess the final product and what form 
should that assessment take? 

• Do we need to have the same model for all disci- 
plines and universities within the one country? 

• Can the assessment be a more meaningful and useful 
learning experience for the candidate, supervisor, 
examiner and institution? 

Background 

An often-quoted adage in higher education goes 
along the lines of: ‘Assessment drives learning’, or, put 
another way: ‘If you want students to learn something 
then you need to assess it’. Perhaps with a more posi- 
tive spin one could say: If it is important enough to 
teach then it is important enough to assess’. However 
it is expressed, assessment is considered an integral 
part of learning and, it is argued, should be aligned 
with what the proposed learning outcomes are for 
students (Biggs, 1999;Toohey, 1999). In addition to the 
‘assessment drives learning’ concept is the idea that 
assessment should be a meaningful part of learning 
(Biggs, 1997; Boud, 2003). Therefore, this paper takes 
as its underlying principles that the final assessment 
processes for doctoral education should be aligned 
with the proposed learning outcomes and should, in 
some way, contribute to the learner’s development and 
understanding of being a researcher. 

Of course, to articulate the learning outcomes of 
the doctorate adequately is far from easy, even at the 
broadest level. Perhaps because of the complex nature 
of doctoral education most universities have settled 
for a list of graduate attributes, with a high level of 
consistency across universities with the following as 
an example: working in teams, time and project man- 
agement, effective use of information and communica- 
tions technology, critical thinking, analytical abilities, 
problem solving, and ethical approaches to research 
(Kiley, 2006). 

Traditionally the examination of doctoral theses in 
Australia, while varying slightly between universities 
and disciplines, is similar to the following. Early in can- 
didature, candidates present a proposal seminar on 
which formative feedback is provided. In some uni- 
versities, this proposal seminar is also considered in a 
summative fashion with candidates being asked to exit 
the programme if their work to date is judged to be 
less than ideal. Supervisors and advisors also provide 
ongoing formative feedback on work as it develops 
and on the candidate’s progress. In many universities 


there is some form of mid-term presentation in which 
candidates receive feedback from supervisors and the 
wider department. The written dissertation, usually a 
maximum of 100,000 words, is the single final item of 
examination except in the Performing and Fine Arts, 
where the performance or exhibition and an exegesis 
provide the basis for examination; and in a Profes- 
sional Doctorate, where coursework in the first year is 
assessed, and then the written dissertation is assessed 
as per the PhD (see Council of Deans and Directors of 
Graduate Studies in Australia, 2005). 

The final written dissertation is sent to two (or 
three) external examiners with ‘external’ meaning 
external to the university. Approximately 50 per cent 
of all Australian theses are examined outside Australia 
and approximately 50 per cent of universities use 
three examiners (Bourke, 2005). In most universities 
the names of examiners are not initially disclosed to 
candidates or to the other examiners, although candi- 
dates are usually involved in discussing a list of poten- 
tial examiners. Each examiner, without contact with 
co-examiners, prepares a written report (on average 
3-4 pages) providing formative feedback to the can- 
didate, supervisor and/or university. Each examiner 
also provides a recommendation to the university’s 
higher degrees committee (or similar) as to whether 
the dissertation should be ‘accepted as is’, ‘accepted 
with minor editorial, or with more substantial edito- 
rial changes’, ‘re-submitted for examination after sub- 
stantial change’, or considered unacceptable’ Some 
universities allow the examiner the option of suggest- 
ing that the student be awarded a Masters degree in 
lieu of the PhD. Most universities allow examiners to 
request an oral examination of the candidate (and in 
some cases the candidate can request this), although 
this is not common practice. The university’s relevant 
committees discuss the examiners’ recommendations, 
and then make the final decision as to the outcome of 
the examination process. 

Taking an historical perspective on doctoral educa- 
tion it is easy to understand why it is that in Australia 
external assessment practices for doctoral theses were 
adopted. The first doctorate in Australia was offered 
in 1946 and it is not surprising that it was considered 
essential to have such work examined externally (at 
that stage it was taken to mean overseas) as that is 
where many of the ‘experts’ of the time were research- 
ing (Pearson, 2005). This practice was also, no doubt, 
exacerbated by Australia’s colonial ties with Britain. Fur- 
thermore, given Australia’s distance from the sources 
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of such examiners, bringing external examiners to 
engage in an oral examination was out of the question, 
hence the assessment process relied on a written dis- 
sertation and written examiners' reports. Of course 
the context has changed substantially in Australia 
since 1946, including the concentration of a substan- 
tial number of highly qualified and skilled researchers 
within Australia, outstanding developments in interna- 
tional travel, and enhanced communication technolo- 
gies. In recent years, a number of Australian universities 
have been recognised as being among the best in the 
world with three Australian universities in the top 100 
universities listed on the Shanghai Jiao Tong Index, a 
further three in the top 100-200 and three more in the 
top 200-300 meaning that at least nine of Australia’s 37 
publicly funded universities can be considered as good 
as, if not better than, those to which theses might have 
been sent in the past (Institute of Higher Education of 
the Shanghai Jiao Tong University, 2008). Furthermore, 
Australia is acknowledged as a world leader in doc- 
toral education and research (Taylor & Beasley, 2005, 
p. 218). It could be argued, therefore, that the current 
dissertation examination system within Australia is not 
based on current educational practice, but rather his- 
torical circumstances. Despite that, there are several 
observable benefits of the existing system, including: 

• External, and generally international, validation of 
candidates' work (Bourke et al. 2004). 

• A level of objectivity with the examiners being 
unknown to the candidate, and to one another 
(Kiley & Mullins, 2004; Mullins & Kiley, 2002). 

• The argument that the system produces reliable 
results regarding the quality of the thesis (Bourke 
et al. 2006). 

• The provision of written reports from the examiners 
which are often extensive and generally of consid- 
erable value in terms of contributing to the further 
refinement and development of the research (John- 
ston, 1997). 

As with any curriculum, however, the issues related 
to doctoral assessment are: What is the purpose of 
the particular form of education? What is it that we 
want the graduate to know, to be able to do, and to 
be’? Through answering these questions we can then 
ask: How can we appropriately assess this? The argu- 
ment for reviewing and possibly amending a doctoral 
examination system relates to aligning the assessment 
with the learning and research experiences of the con- 
temporary doctoral candidate, an argument which is 
expanded below. 


Purpose of the doctorate 

A topic of frequent discussion among those involved 
in research education is ‘What is the purpose of the 
doctorate?’ and more specifically ‘What is the pur- 
pose of the dissertation?’ A particular insight into the 
contemporary purpose of the doctorate is the sub- 
stantial increase in the focus on developing research 
and employability skills within the doctorate demon- 
strated through an original contribution to knowledge. 
For example, in addressing the changing roles of the 
doctorate in Japan, Yamamoto (2008) states: 

Globalisation and changes in industrial structure 
are leading to changes in the demands for profes- 
sionally and academically trained workers. One 
national problem is that existing graduate pro- 
grammes do not fully respond to those needs... In 
Japan, outside of academia, PhDs are still regarded 
as a special kind of people who are difficult to 
manage. . .Overcoming this belief and giving PhDs 
important roles, I think, are the solutions that will 
respond to globalization, (p. 215) 

Referring to Germany, Enders (2005) argues: 

PhD graduates have a significant career advantage 
[outside higher education] in comparison with their 
graduate peers from the same discipline without a 
PhD. Furthermore, the PhD provides an entrance 
ticket especially for elite positions — consider, for 
example, that 50 per cent of the members of the 
board of the 200 biggest German companies have a 
PhD. (2005, p. 122) 

From her extensive study of US academics engaged 
in doctoral education Lovitts (2007) suggests that they 
saw the purpose of the dissertation being both proc- 
ess and product. The process included research train- 
ing and the demonstration of the mastery of research 
skills, and the product included the credentialling of 
the candidate and their contribution. 

In Australia, the idea of the doctorate being mainly 
about completing a substantial body of original work 
was challenged with the introduction of time limita- 
tions when the Australian Government brought in 
the Research Training Scheme (RTS) in the late 1990s. 
This scheme provides funding to universities for a 
maximum of four years for the education of doctoral 
candidates. The majority of the funds are provided on 
student completion, not on enrolment (Gordon, 2000; 
Neumann, 2007). 

A more recent Australian Government develop- 
ment which indicates even more strongly the shift to 
seeing the doctorate as training future researchers’ is 
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the new reporting mechanisms for Australian universi- 
ties with regard to Higher Degree by Research (HDR) 
programmes. Previously reporting to the Department 
of Education, Science and Training (DEST), universities 
now report on this aspect of their education provision 
to the Department of Innovation, Industry, Science 
and Research (DIISR 2008). This Government depart- 
ment’s website states ‘DIISR strives as a key priority 
to encourage the sustainable growth of Australian 
industries by developing a national innovation system 
driving knowledge creation, cutting edge science and 
research, international competitiveness and greater 
productivity' .This move suggests that the Government 
clearly sees doctoral education as contributing to Aus- 
tralia’s innovation agenda. 

Such a view has implications for doctoral pedagogy 
and curriculum, including 
assessment. One change of 
focus is argued by Cham- 
baz (2008, p. 19) where he 
suggests that any focus on 
research training should 
not imply that research 
graduates are merely prac- 
tising to be researchers, but rather they are engaging 
in real’research: 

The European Union. . .established a Charter of 
European Researchers which says that, ‘Doctoral 
candidates should be considered as early stage 
researchers, and so recognised as professionals 
with commensurate rights and duties’. And the 
definition of the ‘early stage researchers’ is the first 
year of practising research including the thesis. It’s 
really that doctoral candidates are considered at the 
European level as ‘early stage researchers”. 

Furthermore, recent developments suggest that the 
focus has shifted somewhat from the ‘original’ con- 
tribution to knowledge towards rather a significant 
contribution to knowledge. From her study of US aca- 
demics Lovitts (2007) suggests that what constitutes 
‘contribution’ is interesting with the question; is a sig- 
nificant contribution sufficient or does it need to be 
an original contribution? This was an area of debate 
among her respondents. But what do examiners look 
for? Isaac, Quinlan and Walker (1992) suggest that 
some examiners look for a demonstration of the can- 
didate’s research skills, whereas others look for the 
training of such skills, and others seek a contribution 
to knowledge. Their work further suggests that aca- 
demics working in laboratory settings are more likely 


to look for training’ whereas those in individualistic 
research environments will more likely look for the 
demonstration’ of skills. 

Certainly work by Mullins and Kiley (2002) indi- 
cated that experienced examiners are aware of their 
own views as to whether they are examining the dis- 
sertation as a ‘polished and cohesive piece of writing 
that provides an original contribution to knowledge’ 
or whether they are judging if the candidate has dem- 
onstrated that she/he is ‘capable of undertaking inde- 
pendent research’. 

We received two distinctly different answers, reflect- 
ing, we believe, two quite different views of what they 
were examining. One view was that it is the thesis, 
as a complete and comprehensive document that 
will remain on the library shelf, that is being exam- 
ined, the other argument 
put forward was that it is 
the student as a potential 
researcher who is being 
examined, (p. 382) 
Assuming that, in Aus- 
tralia at least, the belief is 
that the purpose of the 
doctorate is to both make an original contribution to 
knowledge and as a foundation in research training, 
then the following comments by Chubb (2000) are 
apposite: 

The issue of what constitutes a PhD or other higher 
degree by research is something that perhaps should 
be discussed system-wide. No university could shift 
from present expectations on its own — given that our 
staff examine each other’s students — without poten- 
tially disadvantaging our students. If something were 
to be done, if expectations about the quantum of work 
were to be modified, it would need to be done by most 
of our universities. Now, I am not suggesting that the 
requirement for originality, intellectual rigour and so 
on would change, or that the body of work should be 
other than substantial — and obviously I am not pro- 
posing to change the requirement for external exami- 
nation of PhD theses. I am talking about how much a 
student needs to do to prepare them for their role as 
the next generation of researcher. I suspect that close 
examination could reveal that the expectations are 
now too high. (p. 19) 

Accepting that there is both a role for an original 
contribution to knowledge, and for research education 
within the doctoral programme, this leads us to ask: 
What is the object of assessment? 


... the belief is that the purpose of the 
doctorate is to both make an original 
contribution to knowledge and as a 
foundation in research training, 
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The object of assessment 

Based on the argument above a slightly different way 
of defining the purpose of the doctorate is to suggest 
that it aims to educate future researchers by having 
them undertake a substantial, supervised, research 
project which makes a contribution to knowledge. In 
many, but by no means all cases these researchers will 
become the next generation of academics (Neumann, 
et al. 2008). Whether the graduate goes into an aca- 
demic career at some stage or not, in the final assess- 
ment of the doctoral candidate we are considering: 

• The skill development of the candidate. 

• The candidate's demonstrated potential to under- 
take research, and 

• The quality and originality of the contribution to 

knowledge that the candidate has made throughout 

the research project. 

While most universities provide programmes and 
support for candidate skills development, not all univer- 
sities explicitly acknowledge that many research can- 
didates commence their degree with highly advanced 
professional employability skills (e.g. the mature-age 
professional undertaking a part-time research degree, 
or the candidate who commences a doctorate having 
undertaken a well structured research learning experi- 
ence in their Honours or Research Masters programme). 
Despite the entry criteria and level of knowledge and 
skill, in curriculum terms, at the doctoral examination 
stage, we are considering the outcomes, not necessar- 
ily the inputs, of the programme given that the examin- 
ers do not know the candidate, and unlike academics 
in Lovitts’ (2007) study, are unable to make judgments 
regarding the development or otherwise of such skills 
prior to, or during candidature. 

As the work of Ryland (2007) and the University 
of Queensland Social Research Centre (2007) pro- 
pose, the transfer of skills that have been developed 
through employment to the research education expe- 
rience was common. Hence, skill development, it is 
argued, is by no means a one-way process from the 
research education experience to the workplace or 
society. 

Being able to present an argument and to answer 
questions on one’s area of expertise are generally con- 
sidered to be skills of a research graduate. Clearly in 
many cases the assessment of such skills can only occur 
in an authentic setting where the candidate presents 
her/his work to an audience. On these grounds, the 
argument for some sort of oral assessment toward the 


end of candidature is quite compelling.As Kiley (2006, 
p. 166) suggests: 

These skills, on the whole, are not formally 
assessed, other than through the examination of 
the thesis, which in Australia rarely includes an 
oral exam. Leaving aside the question of whether 
the attributes that universities have identified are 
the most appropriate, the question arises, should 
universities even attempt to assess such skills? If 
so, how might they be assessed so that they lead 
to doctoral student learning, not simply assessment 
for its own sake? 

With regard to the second area of assessment, that 
is the candidate's capability for undertaking research, 
some of the examiners in the Kiley & Mullins (2004) 
and Mullins & Kiley (2002) studies suggested that they 
were able to identify a candidate’s research capability 
through the way that she/he presented their findings. 
However, others suggested that this was quite difficult, 
particularly in disciplines where there was consider- 
able teamwork and multiple authors on publications. 
Being able to authenticate the candidate’s contribu- 
tion to the research was one of the main benefits of 
an oral examination proposed by interviewees in the 
study by Tinkler & Jackson (2004, p. 18). 

With regard to the third area of assessment, that is, 
the quality and originality of the research, it is likely 
that a sound argument can be made to support the 
idea that assessment can be effectively managed 
through examination of the dissertation itself. Whether 
the expectation is a substantial contribution to knowl- 
edge as Lovitts (2007) suggests from some of her 
informants, or the three different types of originality’: 
the topic, the process and the outcomes outlined by 
Tinkler & Jackson (2004) — most examiners report that 
they are able to identify such a contribution to knowl- 
edge, even if this is just ‘A spark of inspiration as well 
as perspiration’ as Winter et al. (2000, p. 35) amusingly 
define originality. 

Assessment decisions: How and by whom? 

The issue of the form of assessment is perhaps the 
most vexed for Australian universities. Candidates tend 
not to be supportive of orals as they have either heard 
the ‘horror stories’ that surround the viva voce or 
defence process and/or they have not appreciated the 
range and variety of oral examination processes that 
exist. Many administrators are not overly supportive of 
oral examination, particularly if they have not thought 
through the options of tele/videoconferencing as they 
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generally consider the cost to be exorbitant and organ- 
ization across time zones a logistical nightmare. This 
is despite Australia’s near neighbour, New Zealand, 
having an oral examination where the overseas exter- 
nal examiner, on completion of a thorough reading 
of the thesis, discusses her/his views and comments 
with a local’ examiner. The ‘local’ examiner then puts 
forward the overseas examiner’s comments and ques- 
tions as well as his/her own at the face-to-face oral.This 
process supports the use of overseas examiners while 
at the same time encouraging oral examination in a 
country quite distant from where many of the possible 
examiners are located (Fraser & Rowarth, 2007). 

The argument remains open: internal or external 
and in what proportions? If internals and externals, do 
they all have the same voting rights? If most institu- 
tions select experienced examiners then how might 
inexperienced academics gain doctoral assessment 
skills? Is there room for a representative of industry, 
the professions or the community on the examination 
panel given the focus on life-long and employability 
skill development during doctoral education? 

In many higher education systems there is the 
involvement of an internal examiner to a greater or 
lesser extent. In some cases, the internal’ is the can- 
didate’s supervisor; in other cases it is someone from 
within the department or from a cognate discipline. 
Furthermore, within the range of types of internal 
examiners there is also a range of roles. In some sys- 
tems the role of the internal examiner is to arrange 
and oversee the oral assessment, for example, briefing 
the assessors, chairing the session and coordinating 
the preparation of the final result and report. In others, 
the supervisor is involved in the process but is not an 
assessor but rather explicitly involved as a proponent 
for the candidate and their research (Hartley, 2000). 

The practice of not using the supervisor as an asses- 
sor, but perhaps as a proponent or chair, fits with 
the principle that it is difficult to effectively be both 
advisor/supporter and assessor. One of the reported 
strengths of the current Australian system is that it 
does not include an internal assessor, leading to exter- 
nal, and generally international, scrutiny of quality 
and standards. Furthermore, with the Australian Gov- 
ernment’s funding model which funds on doctoral 
candidate completions, the possibility of universities 
being perceived to ‘soft mark’ their own candidates 
based on financial rather than educational criteria is 
avoided through the sole use of external examiners. 
This also helps to avoid the ethical dilemmas alluded 


to by Lovitts (2007) with regard to conflict of interest 
between internal examiners, candidates and institu- 
tions.Another consideration related to the use of inter- 
nal and external examiners is the weighting given to 
the various views. For example, does the chair of the 
panel have a vote, or maybe only a deciding vote? Is 
the external examiner’s view weighted more heavily 
than the internal examiner’s? Determining the various 
weightings and considerations is almost as important 
as determining the number of internal and external 
examiners to be involved. 

The issue does not just stop with the ‘internal/ 
external’ consideration, but as research has shown 
(Buckridge, 2003; Kiley & Mullins, 2004; Trafford, 
2003; Mullins & Kiley, 2002) there are discernible dif- 
ferences between experienced and inexperienced 
examiners and the way in which they approach 
examination. Generally inexperienced examiners 
tend to focus more on the ‘parts’ than on the whole, 
and the technical detail than the coherent argument, 
compared with their more experienced colleagues. 
These results have now led a number of UK universi- 
ties to require that between them the two doctoral 
examiners must have examined a minimum of four 
doctoral theses (S. Taylor, personal communication, 
December 7, 2006). 

There is little evidence to suggest that the exami- 
nation process itself is better if it involves examin- 
ers from outside the country. Although, in terms of 
prestige, the ‘marketing’ of outstanding candidates 
to potential ‘employers’, and the general ‘interna- 
tional seal of approval’, most Australian universities 
seek to include at least one overseas examiner where 
appropriate. The ‘marketing’ of particularly talented 
graduates is a practice that appears to be well known 
within the academy (Mullins & Kiley, 2002). This is 
especially the case in Australia where a supervisor 
will often recommend that a dissertation be sent to 
someone in a research group who will (a) appreciate 
the quality of the work; and (b) consider offering the 
graduating student a post-doctoral position or refer- 
ence for future positions based on the quality of the 
dissertation. 

If we are considering the doctorate as a means of 
contributing to a country’s research and development 
capacity, as well as being a personal and social contri- 
bution, would it be reasonable to suggest that at least 
one of the assessors be from outside the academy? At 
the moment, at least inAustralia, caution is exercised in 
using examiners from outside the academy as they are 
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generally considered tougher in their recommenda- 
tions. For example, arising from the Mullins and Kiley 
(2002) study: 

I spent the first 10 years of my career in [a research 
institute] and I had examined theses before I became 
a supervisor. . .Coming from the pure research envi- 
ronment you judge much more against scientific 
papers because that is your benchmark because of 
course what I had done before being a PhD exam- 
iner was to be a reviewer of papers so you take 
very much your benchmark from that [and so they 
are] judging against their peers, someone who has 
been publishing for 20 years. (Sc/Female/24) 

Yet, if we wish to align the assessment with the aims 
of the programme, then might not this be a reasonable 
consideration? 

Should all universities and disciplines 
follow the same method of assessment? 

While there is already considerable variation across 
the academy in the way that doctoral theses are 
assessed, and mindful of the exhortation from Chubb 
(2000) that no one Australian university would be wise 
to change its system without regard to the impact of 
such a change, the question remains, do all Australian 
universities and even disciplines have to adopt the 
same system? Certainly there is considerable disci- 
pline variation already and it might well be that some 
of the professional disciplines might prefer to ‘agree 
to differ'. However, this would be wise only when the 
variation mirrored that which is occurring internation- 
ally. The issue of multi-disciplinary/cross-disciplinary 
theses and their examination would need careful con- 
sideration in these circumstances. 

Having suggested that different disciplines might 
like to approach the examination of doctoral theses 
in discipline-specific ways is not as strange as it might 
sound. Work by Bourke et al. (2004) and Mullins and 
Kiley (2002) suggest that while different disciplines 
have their own particular ways of training and develop- 
ing their researchers, when it comes to examination it 
is often difficult to detect any disciplinary differences, 
other than with technical language, in the reports and 
the ways in which examiners have gone about the 
process. Lovitts (2007) reports a similar phenomenon 
in that US academics in different disciplines ‘often 
used the same words and phrases to describe disserta- 
tions and components of dissertations at the different 
quality levels’ (p xiii) and yet the training process can 
be significantly different. 


E R S I T I E S 

Another form of variation that might be taken into 
account in assessment might be the candidate's pur- 
pose in undertaking a doctorate. For example, could 
a candidate aspiring to be an academic undergo a dif- 
ferent examination process from someone who plans 
to work within government or private enterprise? The 
notion of alignment would suggest that different forms 
of assessment might be appropriate if candidates 
engaged in different learning experiences through 
their programme and that there were different aims for 
the programme outcomes. Consideration of this form 
of variation is particularly pertinent with the increased 
focus on the explicit inclusion of generic/employabil- 
ity skills within doctoral programmes. 

More complex is the issue of whether all universities 
within the one country need to follow a similar assess- 
ment method at the doctoral level or is there room for 
variation? Certainly, some universities could argue that 
the quality of their staff and their research reputation 
is such that they could consider adopting a system of 
assessment that involved some use of internal exami- 
nation in the final decision. Furthermore, some of 
these universities are likely to have funding and collab- 
orative research arrangements that make international 
travel for assessment purposes more viable. Australia’s 
Group of Eight universities (Go8), which collectively 
graduate well over 50 per cent of all Australia’s doc- 
toral candidates, might be in such a position. This is 
not to imply that others could not, or that these uni- 
versities would only examine one another’s doctoral 
candidates. Suffice to say that thought could be given 
to the idea of different university groupings adopting 
alternative strategies for assessing doctoral theses. 

Having suggested that there might be reasons that 
justify variability, one might also argue along the lines 
of the BolognaAgreement (Bologna Declaration, 1999). 
One of the main reasons for the Agreement was to 
enable greater mobility of learners and graduates with 
transferable and internationally recognized awards. 
The question is then, to what extent does variability 
across institutions and countries militate against trans- 
ferability and mobility of arguably the academy’s most 
globally recognized award? 

Making assessment meaningful 

With regard to making the assessment process a more 
meaningful learning process for students, the issue of 
timing is important. For example, what would be the 
ramifications of introducing one of the external asses- 
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sors to the candidate’s supervisory panel prior to final 
submission? In other words, might the external asses- 
sor be more involved in giving formative advice to the 
candidate prior to finalizing the dissertation? This could 
be either an Australian/New Zealand external or an 
overseas external who is visiting or via teleconference. 
There are obvious advantages in this model in terms of 
greater use of expert feedback while the candidate is 
still involved in the development of the research. 

On the other hand, one perceived disadvantage 
might be the ‘contamination’ of the final result where 
objectivity is considered a hallmark of the assessment 
system. Perhaps a third option, the introduction of an 
assessment panel or committee could be considered. 
Taking from the US model, and based on practices in 
some Australian universities, departments and schools 
could consider establishing 
a committee or panel, sepa- 
rate from the supervisors, 
who would be involved 
in assessing the thesis 
proposal at the six to 12 
month milestone, a mid- 
term review where such 
practices are in place, and 
a pre-submission oral pres- 
entation as outlined below. 

This model would maintain the current objectivity of 
the external examiners while providing formal, forma- 
tive (and even summative) feedback from staff in addi- 
tion to the supervisors. 

Another aspect of timing is when a final oral pres- 
entation might be most appropriately made. In sys- 
tems that currently involve an oral examination e.g. 
the viva voce in the UK, the oral assessment comes 
after the examiners have read the dissertation and pre- 
pared their reports. In the Canadian system, if the writ- 
ten reports prior to the oral assessment suggest that 
the student may not be successful then generally the 
oral is postponed until the candidate has been able to 
undertake more work (Hall, 2006). In light of interna- 
tional practices, what would it mean in terms of stu- 
dent learning if the oral presentation were conducted 
prior to submission as an approval to submit' process? 

Based on a modification of the purposes for an oral 
assessment outlined by Tinkler & Jackson (2004, pp. 
17-22) it is possible to provide assessors with aims for 
the pre-submission oral presentation, which include: 

• Authenticate that the work being presented is 

indeed the candidate’s. 


• Determine if the candidate can locate her/his 
research within a broader context. 

• Invite the candidate to defend her/his thesis. 

• Assess oral and presentation skills. 

• Clarify areas of weakness and ambiguity. 

• Determine whether the candidate’s work is of suffi- 
cient quality to be finalised and submitted for exter- 
nal examination. 

One benefit of the oral examination not included 
in the Tinkler & Jackson (2004) list is the provision of 
advice to the candidate on what is required to develop 
and refine the final ‘product’ for examination. Further- 
more, one purpose outlined in their work and not 
noted in this paper is to make a final decision on bor- 
derline work; this, in the Australian system would be 
left to the external examiners. 

If a candidate is able to 
articulate a well-researched 
argument and demonstrate 
other high level knowledge 
and attributes, then the 
assessors are in an excel- 
lent position to assess the 
student's learning through 
the project. Whether this 
presentation for approval 
to submit’ is private i.e. 
involving the candidate, assessment panel and admin- 
istrative support only, or opened more widely to other 
staff and candidates, is a matter for discussion. How- 
ever, it is proposed by those universities seeking to 
implement, or having recently implemented such a 
process that it might lead to fewer borderline exami- 
nation cases per institution than currently is the case. 
The reasons put forward for this are that having their 
work exposed to a critical audience, other than the 
supervisors, and having to articulate an argument, or 
in the case of Europe and the USA defend their thesis’ 
candidates might be assisted in refining their thesis 
prior to submission. Such a pre-submission oral pres- 
entation could play a particularly useful role where 
regular, external presentations by doctoral candidates 
at national and international conferences and meeting 
are not the norm 

Following the assessment process outlined above 
the candidate (for all cases other than ‘revise and 
resubmit’) could revise the dissertation in light of the 
formative comments made by the assessment panel. 
This might take up to three months or even longer if 
the research assessed were deemed under-developed. 


Taking from the US model, and based on 
practices in some Australian universities, 
departments and schools could consider 
establishing a committee or panel, separate 
from the supervisors, who would be 
involved in assessing the thesis proposal at 
the six to 12 month milestone... 
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The candidate is not ‘just fixing up a few bits and 
pieces here and there' nor has she/he left the univer- 
sity (and even the country) upon submission of the 
dissertation for examination, as is often the current sit- 
uation in Australia, but is using the formative feedback 
as a means of (perhaps substantially) developing and 
refining the work. There are several clear advantages 
to involving the supervisor/ supervisory panel or com- 
mittee in the oral assessment. For example, it ensures 
that the supervisors have read the dissertation and 
have engaged with the candidate in determining the 
appropriate time for its submission.A further benefit is 
that it allows opportunities for inexperienced supervi- 
sors/assessors to learn through being involved in the 
process with more experienced colleagues. 

When the candidate, in conjunction with the super- 
visory panel, believes that she/he has revised the 
dissertation in light of the comments made by the 
assessment panel, the dissertation would be submit- 
ted to two external assessors. As an additional benefit 
to learning, candidates would be asked to prepare a 
response to the assessment panel’s comments akin to 
the response one makes regarding changes made in 
light of reviewers’ comments on journal manuscripts. 
This response could accompany the revised disserta- 
tion when submitted for final examination. 

Conclusion 

This paper has outlined the issues that need to be 
considered regarding possible changes to the cur- 
rent assessment practices for Australian doctoral can- 
didates. It has also outlined some modest changes, 
which, it is argued, will enable the assessment proc- 
ess to be more closely integrated with learning for the 
candidate, supervisors, examiners and the institution. 
It is suggested that the current system in Australia was 
appropriate at the time the first doctorate was intro- 
duced into Australia in 1946, but developments over 
the past 60 years require rethinking of current curricu- 
lum practices. 

The suggestion has been made that there are ben- 
efits to candidates, supervisors, examiners and institu- 
tions in terms of their learning with the use of oral 
assessment. Furthermore, by careful timing of the oral 
assessment it might be possible to enable formative 
feedback to be a stronger outcome of the assessment 
process compared with the current system where 
examiners’ comments often come too late for mean- 
ingful learning. 


The argument for considering changes to a system 
that in many ways works well is based on the valuable 
learning outcome for candidates, supervisors, examin- 
ers and institutions in ways that align the assessment 
strategies with the aims of the project. 

Dr Margaret Kiley is Convenor, Higher Degree Research at 
Australian National University, Canberra, Australia. 
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